112 research outputs found
2,2-Dibromo-N-(4-fluorophenyl)acetamide
In the crystal structure of the title compound, C8H6Br2FNO, C—H⋯O and N—H⋯O hydrogen bonding results in six-membered rings and links the molecules into chains running parallel to the c axis. The dihedral angle between the fluorophenyl ring and the acetamide group is 29.5 (5)°
Invertible Rescaling Network and Its Extensions
Image rescaling is a commonly used bidirectional operation, which first
downscales high-resolution images to fit various display screens or to be
storage- and bandwidth-friendly, and afterward upscales the corresponding
low-resolution images to recover the original resolution or the details in the
zoom-in images. However, the non-injective downscaling mapping discards
high-frequency contents, leading to the ill-posed problem for the inverse
restoration task. This can be abstracted as a general image
degradation-restoration problem with information loss. In this work, we propose
a novel invertible framework to handle this general problem, which models the
bidirectional degradation and restoration from a new perspective, i.e.
invertible bijective transformation. The invertibility enables the framework to
model the information loss of pre-degradation in the form of distribution,
which could mitigate the ill-posed problem during post-restoration. To be
specific, we develop invertible models to generate valid degraded images and
meanwhile transform the distribution of lost contents to the fixed distribution
of a latent variable during the forward degradation. Then restoration is made
tractable by applying the inverse transformation on the generated degraded
image together with a randomly-drawn latent variable. We start from image
rescaling and instantiate the model as Invertible Rescaling Network (IRN),
which can be easily extended to the similar decolorization-colorization task.
We further propose to combine the invertible framework with existing
degradation methods such as image compression for wider applications.
Experimental results demonstrate the significant improvement of our model over
existing methods in terms of both quantitative and qualitative evaluations of
upscaling and colorizing reconstruction from downscaled and decolorized images,
and rate-distortion of image compression.Comment: Accepted by IJC
Your Transformer May Not be as Powerful as You Expect
Relative Positional Encoding (RPE), which encodes the relative distance
between any pair of tokens, is one of the most successful modifications to the
original Transformer. As far as we know, theoretical understanding of the
RPE-based Transformers is largely unexplored. In this work, we mathematically
analyze the power of RPE-based Transformers regarding whether the model is
capable of approximating any continuous sequence-to-sequence functions. One may
naturally assume the answer is in the affirmative -- RPE-based Transformers are
universal function approximators. However, we present a negative result by
showing there exist continuous sequence-to-sequence functions that RPE-based
Transformers cannot approximate no matter how deep and wide the neural network
is. One key reason lies in that most RPEs are placed in the softmax attention
that always generates a right stochastic matrix. This restricts the network
from capturing positional information in the RPEs and limits its capacity. To
overcome the problem and make the model more powerful, we first present
sufficient conditions for RPE-based Transformers to achieve universal function
approximation. With the theoretical guidance, we develop a novel attention
module, called Universal RPE-based (URPE) Attention, which satisfies the
conditions. Therefore, the corresponding URPE-based Transformers become
universal function approximators. Extensive experiments covering typical
architectures and tasks demonstrate that our model is parameter-efficient and
can achieve superior performance to strong baselines in a wide range of
applications. The code will be made publicly available at
https://github.com/lsj2408/URPE.Comment: 22 pages; NeurIPS 2022, Camera Ready Versio
M-OFDFT: Overcoming the Barrier of Orbital-Free Density Functional Theory for Molecular Systems Using Deep Learning
Orbital-free density functional theory (OFDFT) is a quantum chemistry
formulation that has a lower cost scaling than the prevailing Kohn-Sham DFT,
which is increasingly desired for contemporary molecular research. However, its
accuracy is limited by the kinetic energy density functional, which is
notoriously hard to approximate for non-periodic molecular systems. In this
work, we propose M-OFDFT, an OFDFT approach capable of solving molecular
systems using a deep-learning functional model. We build the essential
nonlocality into the model, which is made affordable by the concise density
representation as expansion coefficients under an atomic basis. With techniques
to address unconventional learning challenges therein, M-OFDFT achieves a
comparable accuracy with Kohn-Sham DFT on a wide range of molecules untouched
by OFDFT before. More attractively, M-OFDFT extrapolates well to molecules much
larger than those in training, which unleashes the appealing scaling for
studying large molecules including proteins, representing an advancement of the
accuracy-efficiency trade-off frontier in quantum chemistry
One Transformer Can Understand Both 2D & 3D Molecular Data
Unlike vision and language data which usually has a unique format, molecules
can naturally be characterized using different chemical formulations. One can
view a molecule as a 2D graph or define it as a collection of atoms located in
a 3D space. For molecular representation learning, most previous works designed
neural networks only for a particular data format, making the learned models
likely to fail for other data formats. We believe a general-purpose neural
network model for chemistry should be able to handle molecular tasks across
data modalities. To achieve this goal, in this work, we develop a novel
Transformer-based Molecular model called Transformer-M, which can take
molecular data of 2D or 3D formats as input and generate meaningful semantic
representations. Using the standard Transformer as the backbone architecture,
Transformer-M develops two separated channels to encode 2D and 3D structural
information and incorporate them with the atom features in the network modules.
When the input data is in a particular format, the corresponding channel will
be activated, and the other will be disabled. By training on 2D and 3D
molecular data with properly designed supervised signals, Transformer-M
automatically learns to leverage knowledge from different data modalities and
correctly capture the representations. We conducted extensive experiments for
Transformer-M. All empirical results show that Transformer-M can simultaneously
achieve strong performance on 2D and 3D tasks, suggesting its broad
applicability. The code and models will be made publicly available at
https://github.com/lsj2408/Transformer-M.Comment: 20 pages; ICLR 2023, Camera Ready Version; Code:
https://github.com/lsj2408/Transformer-
- …